Image labeling systems are used in many image processing application domains such as 3D reconstruction, stereo, object segmentation and optical flow. Image elements such as pixels, groups of pixels, voxels or groups of voxels are labeled by the system as belonging to one of a plurality of specified types. For example, in the case of object segmentation an image element may be labeled as either a foreground or a background image element. This is useful in digital photography, medical image analysis, and other application domains where it is required to find a boundary between a main object in the image and a background. The main object and the background may then be processed separately. In the case of a medical image it may be required to segment out a region of an image depicting a tumor or organ such as the lungs in order to enable a surgeon to interpret the image data.
Existing energy optimization algorithms for image labeling are computationally intensive and time consuming. Such techniques are used for low resolution images and/or where the application domain allows long processing times. However, in recent years advances in image acquisition technologies have significantly increased the size of images and 3D volumes. For instance, the latest commercially available cameras can capture images with almost 20 million pixels. In fact it is now possible to capture giga-pixel images of complete cities. Similarly, medical imaging systems can acquire 3D volumes with billions of voxels. This type of data gives rise to large scale optimization problems to find a labeling which are extremely computationally expensive to solve and require large amounts of memory.
Multi-scale processing is one type of approach that has been used to reduce the memory and computational requirements of optimization algorithms for image labeling. In order to label a large image (or 3D volume) they first solve the problem at a low resolution, obtaining a coarse labeling of the original high resolution problem. This labeling is refined by solving another optimization on a small subset of the pixels. However, such multi-scale approaches are best suited to “blob-like” segmentation targets and tend to fail to segment thin structures. This means that the multi-scale approach is often not suitable for analyzing medical images for example, where fine detail structures may be extremely important when making a medical diagnosis. Also, for other applications such as digital image editing and object recognition thin structures need to be successfully segmented.
There is a need to provide an improved image labeling system which has reduced computational requirements, improved speed and yet gives high quality results.
The embodiments described herein are not limited to implementations which solve any or all of the disadvantages of known multi-scale image labeling systems.